Image capture having temporal resolution and perceived image sharpness

ABSTRACT

A camera for capturing video images in a series of frames includes an image sensor having an array of pixels. Each pixel receives an image and accumulates an electrical charge representative of the image during a frame. The camera also includes a pixel processor to sample a pixel output for each of the pixels of the image sensor during an intermediate portion of the frame to produce a signal representative of the image.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application Ser.No. 62/092,110, entitled “IMPROVED TEMPORAL RESOLUTION AND PERCEIVEDIMAGE SHARPNESS” and filed on Dec. 15, 2014, which is expresslyincorporated by reference herein in its entirety.

BACKGROUND

Field

The present disclosure relates generally to image data. Aspects may beincorporated into cameras and may be used, e.g., to generate images forultra high definition (UHD) displays.

Background

Ultra high definition (UHD) displays are used in a variety ofapplications, including televisions, personal computers and tablets,smart phones, and camera viewfinders. Reduced sharpness in such displaysmay occur due to relative motion of objects in the image. For cameraviewfinders, a “motion blur” effect is caused by relatively longexposure time of each image (20 ms in a 50 Hz system and 16.6 ms in a59.94 Hz system). As an object moves during an image exposure of acamera imager, details are lost as the object is spread over multiplepixels. One solution is to increase the number of frames that are beingcaptured per second, reducing the exposure time of each individual frameas a consequence. All of these frames at the higher frame rate may betransmitted to the display to deliver sharper images but with greatexpense of signal bandwidth. In addition, as the camera generates moreimages per second, the amount of photons received by the image sensorper image diminishes, resulting in a lower light sensitivity of thecamera and lower signal to noise ratio.

As an alternative to creating more images per second, the exposure couldalso be reduced by using an electronic shutter, to keep the number offrames per second the same and only shorten the exposure time of eachimage. This reduces the motion blur (as there's less time for movement)but also leads to a lower light sensitivity and in addition introduces adisturbing strobe effect (i.e., “judder”). Using this approacheffectively produces zero exposure (i.e., blackness) within the framewhen the shutter is closed, creating a gap between the images. The humaneye will attempt to ‘track’ the object motion from frame to frame. Aball that flies through the air, could be razor sharp in each image, butas the camera image sensors only have captured short exposure moments,the visual trajectory information is lost and the ball seems to jumpthrough the scene, lacking smoothness of motion. As the human visualsystem is ‘trained’ to track objects, this leads to a distracted viewingexperience.

SUMMARY

The following presents a simplified summary of one or more aspects inorder to provide a basic understanding of such aspects. This summary isnot an extensive overview of all contemplated aspects, and is intendedto neither identify key or critical elements of all aspects nordelineate the scope of any or all aspects. Its sole purpose is topresent some concepts of one or more aspects in a simplified form as aprelude to the more detailed description that is presented later.

Aspects presented herein provide for improved image performance and areduction in image blur by capturing image sensor output at multiplepoints during an exposure. Information from the multiple points may beused in order to generate an improved signal representative of theimage.

Aspects may include a camera, method, apparatus, system, andcomputer-readable medium for capturing video images in a series offrames includes an image sensor having an array of pixels. Each pixelreceives light photons and accumulates an electrical charge in responseto the received photons. A pixel processor samples the electrical chargeaccumulated by each of the pixels at least one time during anintermediate portion of a frame and processes the samples to produce adigital image, thus reducing motion blur on a display.

To the accomplishment of the foregoing and related ends, the one or moreaspects comprise the features hereinafter fully described andparticularly pointed out in the claims. The following description andthe annexed drawings set forth in detail certain illustrative featuresof the one or more aspects. These features are indicative, however, ofbut a few of the various ways in which the principles of various aspectsmay be employed, and this description is intended to include all suchaspects and their equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example camera system 100 having aviewfinder display;

FIG. 2A is block diagram of an example detail correction circuit of apixel processor enhanced by using an intermediate exposure for thedetail correction signal;

FIG. 2B is a block diagram of an example subtractor for determiningmultiple intermediate exposures;

FIG. 3 a graphical illustration of example pixel charge rates during asequence of exposures as an indication of motion presence;

FIG. 4 is a flowchart of an example method for applying an intermediateexposure for the detail correction signal;

FIG. 5A is block diagram of an example motion blur reduction circuitthat switches to a pixel output of an intermediate exposure in responseto motion detection;

FIG. 5B is a block diagram of an example intermediate exposureprocessor;

FIG. 6 is a flowchart of an example method for switching to a pixeloutput from an intermediate exposure when motion is detected;

FIG. 7 is a graphical illustration of pixel charge rates and pixeloutput for a series of adjacent pixels exposed to an object in motion.

FIG. 8 is a block diagram of a detail correction circuit that combinesthe elements of FIG. 2A and FIG. 5A.

FIG. 9 is a graphical illustration of pixel charge output for a seriesof adjacent pixels exposed to an object in motion.

DETAILED DESCRIPTION

The detailed description set forth below in connection with the appendeddrawings is intended as a description of various configurations and isnot intended to represent the only configurations in which the conceptsdescribed herein may be practiced. The detailed description includesspecific details for the purpose of providing a thorough understandingof various concepts. However, it will be apparent to those skilled inthe art that these concepts may be practiced without these specificdetails. In some instances, well known structures and components areshown in block diagram form in order to avoid obscuring such concepts.

Certain aspects of video production systems will now be presented withreference to various apparatus and methods. These apparatus and methodswill be described in the following detailed description and illustratedin the accompanying drawing by various blocks, modules, components,circuits, steps, processes, algorithms, etc. (collectively referred toas “elements”). These elements may be implemented using electronichardware, computer software, or any combination thereof. Whether suchelements are implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem.

By way of example, an element, or any portion of an element, or anycombination of elements may be implemented with a “pixel processor” thatincludes one or more processors. Examples of processors includemicroprocessors, microcontrollers, image processors, digital signalprocessors (DSPs), field programmable gate arrays (FPGAs), applicationspecific integrated circuits (ASICs), programmable logic devices (PLDs),state machines, gated logic, discrete hardware circuits, and othersuitable hardware configured to perform the various functionalitiesdescribed throughout this disclosure. One or more processors in theprocessing system may execute software. Software shall be construedbroadly to mean instructions, instruction sets, code, code segments,program code, programs, subprograms, software modules, applications,software applications, software packages, routines, subroutines,objects, executables, threads of execution, procedures, functions, etc.,whether referred to as software, firmware, middleware, microcode,hardware description language, or otherwise. The software may reside ona non-transitory computer-readable medium. A computer-readable mediummay include, by way of example, non-transitory storage such as amagnetic storage device (e.g., hard disk, floppy disk, magnetic strip),an optical disk (e.g., compact disk (CD), digital versatile disk (DVD)),a smart card, a flash memory device (e.g., card, stick, key drive),random access memory (RAM), read only memory (ROM), programmable ROM(PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), aregister, a removable disk, as well as a carrier wave, a transmissionline, and any other suitable medium for storing or transmittingsoftware. The computer-readable medium may be resident in the processingsystem, external to the processing system, or distributed acrossmultiple entities including the processing system. Those skilled in theart will recognize how best to implement the described functionalitypresented throughout this disclosure depending on the particularapplication and the overall design constraints imposed on the overallsystem.

FIG. 1 is a simplified block diagram illustrating an example camerasystem 100 that implements a pixel processor 122. Camera system 100 maycomprise a camera 102, the viewfinder 104, and a lens system 106. Camera102 may include an image sensor 120, which may comprise an array ofpixels to convert photons to electrical charges. Among others, the imagesensor may comprise a charged coupled device (CCD) or complementarymetal oxide semiconductor (CMOS). Thus, the array of pixels may comprisean array of CCD or CMOS pixels. An image is projected by the lens system106 onto the image sensor 120. The output of the image sensor 120comprises an output signal from the array of pixels. The image sensor120 produces a voltage signal by converting the photon input level foreach pixel to a proportional voltage signal for each pixel in the array.The pixels of the image sensor 120 are exposed to photons, e.g.,projected by the lens system 106 and read by the pixels of the imagesensor in time units of frame exposures. Each pixel accumulates anelectrical charge representative of the image during the course of theexposure frame. The image sensor 120 may convert the electrical chargeto an analog output voltage signal. Alternatively, the image sensor 120may convert the electrical charge to an analog voltage and convert theanalog voltage to a digital signal using an analog to digital converter,for example, to produce a digital output voltage signal. The imagesensor 120 may transmit the output voltage signal periodically at theframe rate. The pixel may be reset by discharging the accumulated chargeso that the next pixel charge accumulation for the next frame can begin.The amount of light photons may be converted to the voltage signal up toa saturation threshold, at which point no further charge can beaccumulated for the pixel output. In one example, multiple image sensors120 may operate in a synchronous manner. Multiple image sensors 120 mayalso operate in different phases relative to one another.

Pixel processor 121 may be configured to correct the image sensor pixeloutput signals for motion blur. The output of the pixel processor 121may be an array of pixel signals to form an image for each frameexposure of the video sequence. Camera 102 includes a video processor122 that receives a sequence of images and produces a digital videooutput having a desired frame rate, aspect ratio, etc. The videoprocessor 122 may also perform white balance, color correction and gammacorrection to the video images. The video processor 122 may beimplemented as a plurality of separate processors each configured toperform one or more of the above functions. Alternatively, the pixelprocessor 121 and video processor 122 may be arranged in a reversemanner, whereby the pixel processor 121 processes images on a per pixelbasis already corrected by the video processor 122.

An encoder 124 may receive a raw video output from video processor 122and produce a formatted digital video signal encoded according to aparticular specification (e.g., Serial Digital Interface (SDI),H.264/MPEG-4 Advanced Video Coding, or High Definition MultimediaInterface (HDMI)). The signal from encoder 124 may be output fortransmission to a video production system and/or over a network usingtransceiver 126. Encoder 124 may also provide an encoded or raw videofeed to viewfinder 104.

View finder 104 may include a decoder 141 configured to receive encodedvideo or raw video from encoder 124 and provide image data for thedisplay 142. In one example, the display 142 may include an organiclight-emitting diode (OLED) at each pixel, whereby a light-emittingdiode (LED) is coated with an emissive electroluminescent layer formedfrom an organic compound which emits light in response to an electriccurrent. These and other devices may be used to generate images on thedisplay 142.

Lens system 106 may include one or more lenses and may be controlled toprovide a desired optical configuration of lenses, which configurationmay specify, for example, a depth of field setting, a numericalaperture, and a focal length.

FIG. 2A is block diagram of an example detail correction circuit 200 ofthe pixel processor 121 for sampling the image sensor 120 during anintermediate portion of the frame to produce a signal representative ofthe image. For each pixel of the image sensor 120, the sampler 202receives input 201, which is the pixel output received from the imagesensor 120. The sampler 202 may read the cumulative pixel output values(e.g., an electrical charge or output voltage signal values) at discretetime samples S1 to Sn. Samples S1 and Sn may occur at the beginning andend of an exposure frame, respectively, with one or more samplesoccurring between samples S1 and Sn. For example, with n=4, sample Sn=S4occurs at the end of the frame exposure, a sample S2 may occur at about25% of the full frame exposure interval, and a sample S3 may occur atabout 75% of the full frame exposure interval. Alternatively, sample S2may occur at about 33% of the full frame exposure interval, and a sampleS3 may occur at about 66% of the full frame exposure interval. As analternative example, with n=3, sample S3 occurs at the end of theexposure frame, and a sample S2 may occur anywhere between about 25 to75% of the frame exposure. Other possible alternative variations for n>4may be implemented, where sampler 202 provides additional samples ofpixel output values within the full frame exposure. For a CMOSimplementation of image sensor 120, the sampler 202 may operate asdescribed above to read the multiple samples per exposure frame. In thecase of a CCD implementation of image sensor 120, reading intermediatesamples may not be possible within a single frame. Consequently, a CCDimage sensor 120 may need to operate at a faster frame rate than thenominal rate for the camera 100 to simulate multiple samples within thenominal frame. For example, if the video signal for camera 100 isprocessed at rate of 50 FPS (20 ms frames), and sampler 202 requiresfour samples per frame (n=4), then the CCD image sensor 120 may operateat a faster rate of 200 FPS (5 ms frames), yielding 4 CCD images per 20ms frame. Each set of samples S1 to S4 may then be derived from a blockof four CCD images for an equivalent 20 ms frame.

A subtractor 203 determines the cumulative pixel output (e.g.,electrical charge or output voltage signal value) for the full frameexposure by subtracting the pixel output value at sample S1 from thecumulative pixel output value at sample Sn. A subtractor 204 determinesthe cumulative pixel output value of an intermediate frame exposure bysubtracting the cumulative pixel output value at sample S2 fromcumulative pixel output value at sample S3. For the example of n=4,sample S2 at 25% of the exposure frame and sample S3 at 75% of the fullframe exposure interval, the intermediate exposure provides the pixeloutput value for the middle 50% of the frame exposure. For the examplewhere n=4, sample S2 occurring at about 33% of the frame exposure andsample S3 at about 66% of the full frame exposure, the intermediateexposure provides the pixel output for the middle third of the fullframe exposure. Alternatively, for the example of n=3, sample S3 occursat the end of the full frame exposure, and sample S2 at 50% of the fullframe exposure, subtractor 204 may subtract the pixel output value atsample S2 from the pixel output value at sample S3 to provide anintermediate exposure value related to the last half of the full frameexposure. Alternatively, subtractor 204 may subtract the pixel outputvalue at sample S1 from the pixel output value at sample S2 to providean intermediate exposure value related to the first half of the fullframe exposure.

FIG. 2B shows a block diagram for an example subtractor 204, in whichmultiple intermediate exposures may be determined. In an embodiment withsampler 202 reading samples Sn for n>4, multiple intermediate exposuresmay be obtained by subtractor 204 by using multiple subtractors 214 toeach determine the respective intermediate exposure separately. Forexample, for n=6, intermediate exposures S6-S5, S5-S4, S4-S3, S3-S2 andS2-S1 may be determined by one or more subtractors 214. Selector 224 maybe implemented as a multiplexer to adaptively select which intermediateexposure is to be processed by the detail correction circuit 200.

Amplifier 205 receives the pixel output of the intermediate frameexposure and amplifies it as a normalization to a full frame exposure.For example, the amplifier 205 may apply a 6 dB boost to the cumulativepixel output value. A detail processor 206 receives the amplified pixeloutput value and performs a detail correction algorithm to correctmotion blur. The detail processor 206 improves the perceived imagesharpness by generating a correction signal at any signal transition.Transitions in luminance and/or chrominance are emphasized by the detailprocessor 206 to enhance objects in a scene. The calculated detailcorrection is added to the original image on a pixel by pixel basis.This detail correction signal depends on the sharpness of the image. Inthis example, the intermediate exposure contains 50% of the motion blurfor the pixel. By performing detail correction on the shorterintermediate frame exposure instead of the full frame exposure, theeffect of motion blur in the pixel is reduced, which enhances theeffectiveness of the detail correction. Summer 207 is configured to addthe detail correction signal to the full exposure pixel output, givingan enhanced pixel output 208.

FIG. 3 is a graphical illustration of an example sampling of an imagesensor 120 pixel output for multiple samples S1 to Sn during a series ofsingle frame exposures 301, 302, 303. In this example, n=4 andcumulative pixel output values are sequentially read at samples S1, S2,S3 and S4, with samples S1 and S4 providing the cumulative pixel outputfor a full frame exposure as sample S1 occurs at the beginning of theframe, and sample S4 occurs at the end of the frame. A pixel output ofimage sensor 120 for an intermediate frame exposure can be obtainedbetween samples S2 and S3. For exposure 301, the constant slope of theaccumulating pixel output during the interval between S1 and S4indicates a constant photon input for this pixel, which means a constantlight level is being reflected from an object in the camera field ofview. Since the light level is constant, the object is likely fixed andnot moving across this pixel unit within the frame exposure 301. Incontrast, the frame exposures 302 and 303 illustrate pixel charge ratevariation between each sample pair interval (e.g., Si and Si+1). Forexample, in exposure 302, the rate of pixel charge accumulation isconstant between samples S1 and S2, S2 and S3, but falls between samplesS3 and Sn, revealing the presence of motion for the pixel output, from abrighter object to a darker object, and hence the potential for motionblur. Exposure 303 shows the presence of motion as a transition from adarker object to a brighter object being sensed by the pixel of imagesensor 120 (i.e., more light photons being sensed by the pixel), as thepixel charge rate increases between samples S3 and Sn compared to thepixel charge rate between S1 and S3. Thus, with the sampler 202configured to track intermediate exposures within each frame, (e.g.,between samples S2 and S3), motion is detectable. In contrast, aconventional approach which measures pixel output only at the beginningof the frame and at the end of the frame (i.e., at samples S1 and Sn)would give misleading pixel charge rates 312 and 313, and overlook theindication of motion.

FIG. 4 shows a flowchart of an example method 400 to implement thedetail correction circuit 300. In step 402, a pixel output value (i.e.,an electrical charge or output voltage signal value) is read at samplesS1 to Sn by the sampler 302. In step 404, the pixel output for theintermediate frame exposure may be determined by subtractor 204. Forexample, as described in connection with the example of FIG. 3, thepixel output for the intermediate frame exposure may be sampled bysubtracting the pixel output at sample S2 from the pixel output atsample S3. The amplifier 205 may amplify the pixel output value of theintermediate exposure at step 406. Optional aspects are illustrated witha dashed line in FIG. 4. The detail processor 206 performs detailprocessing of the amplified pixel output value of the intermediateexposure at step 408 and produces a detail correction signal 210. Thisdetail correction is enhanced by reducing the exposure of the motion tothe reduced intermediate exposure period compared to performing detailprocessing on the full exposure period. In step 410, the detailcorrection signal 210 may be added to the pixel output value of the fullexposure at summer 207, producing a final pixel output signal 208corrected of motion blur. While detail processing may be based on theintermediate exposure, the pixel output signal 208 captures the fullexposure pixel output plus the detail correction signal 210. Thus, themethod 400 reduces the effect of motion blur in an image sensor pixelwithout any judder that would result using the conventional approach ofsimply shuttering the pixel output for each frame exposure to compressthe motion duration within the frame.

FIG. 5A is block diagram of an example motion blur reduction circuit 500of the pixel processor 121. Input 501 of sampler 202 takes the pixelsignal from image sensor 120, and reads the cumulative pixel output atmultiple samples S1 to Sn. Subtractor 203 determines the full exposurepixel output as explained above with reference to FIGS. 2A and 2B.Subtractor 504 determines the intermediate exposure pixel output in asimilar manner as described above with respect to subtractor 204 inFIGS. 2A and 2B. Comparator 506 is configured to exploit theintermediate exposure sampling to detect motion as shown in exposures302, 303 in FIG. 3. In one embodiment, comparator 506 is configured todetermine the ratio of full exposure duration and intermediate exposureduration. For example, where the interval between samples S2 and S3 is50% of the full exposure interval between samples S1 and S4, comparator506 may determine that the S3-S2 interval is half of the interval Sn-S1,and thus the pixel output during S3-S2 interval is half the pixel outputfor the full frame interval Sn-S1 if the pixel charge rate is constant.The comparator 506 may then compare the pixel output of the intermediateexposure to one half of the pixel output for the full exposure. If thiscomparison is an equal comparison, then the comparator determines thatno motion is present because the expected pixel output has beenconfirmed to be constant (e.g., exposure 301 of FIG. 3 showing aconstant pixel charge rate). If on the other hand the comparison isunequal, then comparator 506 determines that the pixel charge rate isnot constant for this frame, and motion is therefore detected. Since theeffect of motion blur is reduced when the duration of motion is reduced,comparator 506 operates switch 507 to position B, allowing the pixeloutput 508 to be derived from the intermediate exposure. Amplifier 205normalizes the pixel output to be proportional to that of a fullexposure. For the 50% intermediate exposure example, the amplifier 205is configured to boost the pixel output value roughly by 200%. When nomotion is detected by comparator 506, switch 507 is configured to moveto position A in response to a control signal from comparator 506,allowing the full exposure pixel output to be transmitted as pixeloutput 508. Each pixel of the image sensor pixel array may be processedaccordingly to generate the full digital image for the frame. Thus,depending on detection of motion in the pixel and with the switching ofswitch 507, a digital image may be produced having a mix of pixeloutputs 508 of the full exposure and pixel outputs 508 of theintermediate exposure to form the pixel array for the full digitalimage.

FIG. 5B shows an example block diagram of an alternative embodiment forderiving the intermediate exposure pixel output. Instead of subtractor504, an intermediate exposure processor 504′ may be implemented by anintermediate exposure selector 524 which reads the intermediate samplesS14 received from sampler 202, and performs a weighting algorithm whichranks each sample for the amount of motion blur and selects the samplehaving the least amount of blur.

FIG. 6 shows a flowchart of an example method 600 of generating an imagesignal with reduced motion blur. The method may be performed by at leastone processor, such as pixel processor 121 or motion blur reductioncircuit 500. The method may be performed by a camera such as camerasystem 100 to improve an image generated by the camera.

In step 602, sampler 202 reads pixel output at multiple sample points S1to Sn. Although examples have illustrate three or four samples points,any number of samples points may be used in connection with the aspectspresented herein. In step 604, subtractor 204 determines the pixeloutput for the intermediate exposure by subtracting the pixel outputvalue at sample S2 from the pixel output value at sample S3. Next, at606, comparator 506 determines a factor k based on the ratio ofintermediate exposure duration to full exposure duration (e.g., if theduration of the intermediate exposure period is one half the duration offull exposure period, comparator 506 determines that k=½), and performsthe following comparison test:kFE/IE=1,where FE is full exposure pixel output, e.g., Sn-S1 and IE isintermediate exposure pixel output, e.g., S3-S2. If the comparison testis not true, then motion is detected. Motion may be movement from abrighter object to a darker object as in exposure 302 or the oppositemovement is detected as in exposure 303. When motion is detected, at 610switch 507 selects the pixel output for the output 508. At 608, thepixel output for the intermediate exposure may also be amplified byamplifier 205. If the comparison test is true, then no motion isdetected and comparator 506 activates switch 507 to select the pixeloutput from the full exposure at 612.

At 614, digital image data of the image is generated from the pixels,e.g., to produce a signal representative of the image. This may includegenerating a signal based on an array of pixels for an image sensor,such as 120.

The method may further include generating digital image data from thearray of pixels such that a full image for one frame duration includes amix of pixels having selected samples when no motion is detected andselected samples when motion is detected. This may include amplifying amagnitude of the pixel output during the intermediate portion of theframe by a magnitude that balances with a magnitude of the pixel outputfor a full frame and performing detail processing on the amplifiedmagnitude to produce a detail correction signal. The detail correctionsignal may be added to the pixel output for the full frame to producedigital image data corrected for motion blur, when an object in motionis not detected and the detail correction signal may be added to thepixel output of the intermediate portion of the frame, when an object inmotion is detected.

The method may further include determining digital image data for aplurality of adjacent pixels for an object in motion that moves acrossthe plurality of adjacent pixels during one frame duration.

The method may further include selecting at least one sample during theintermediate portion of the frame by selecting a pixel output havingleast amount of blur according to a weighted detail comparison of aplurality of pixel outputs. For example, additional samplings may betaken at different times within an exposure to have either a smaller orlarger effect to the exposure time of the additionally produced signal.This may offer a larger improvement in the resolution and sharpness. Adetail signal may be made for multiple snapshots, e.g., samples and thedetail may be weighted according to an algorithm that detects the mostreliable snapshot, e.g., the sample having the least amount of blur.

In another example, aspects presented herein may also be performed bygenerating more images per second, without the accumulation features inCMOS imagers, and adding up the individual time frames to the imagers.

The pixel processor 121 may comprise at least one processor coupled to acomputer-readable medium/memory. The computer-readable medium maycomprise executable code for causing the at least one processor toperform the aspects of the method illustrated in connection with FIG. 4and/or FIG. 6.

FIG. 7 shows a graphical illustration of an example pixel charge ratefor a series of consecutive pixels of image sensor 120 as an object inmotion 701 moves across the pixels. For simplicity of illustration, inthis example, the object 701 is represented as a simple shape that canmove evenly through the sensing region of each of four pixels, affectingthe photons sensed by each image pixel. It should be understood that anycombination of objects in motion and with various sizes can impact theinput of photons to the image sensor pixels, and the pixel charge ratesmay be more complex as a result. In this example, four pixels P1 to P4are examined, however the process may be applied for the entire array ofpixels in a similar manner on a per row basis, or in blocks of pixels ofa particular number. In this example, sampling is performed by sampler202 at samples S1 to Sn, where n=4, and samples S2 and S3 occur at 25%and 75% of the full exposure time. The object in motion 701 is movingfrom pixel P1 toward pixel P4 for the time duration of a single exposureframe, from time T0 to time T1. As the object moves past each pixel, thepixel charge is activated as shown in pixel rate chart 702 during theexposure T0-T1. Thus, 702 illustrates how the output signal of the pixelwill accumulate over time. For pixel 1, the object activates a pixelcharge for roughly the first 25% of the exposure, and the accumulatedcharge is maintained at a constant level for the remainder of theexposure period reflecting the absence of motion as the object has movedonto pixel P2. At pixel P2, the pixel charge accumulates during thefirst 50% of the exposure frame. The pixel charge at pixel P3accumulates during the middle 50% of the exposure frame, while pixel P4accumulates pixel charge only in the final 25% of the exposure frame.

A pixel output plot 703 across the four pixels for the full exposure isshown having a constant value between time T0 and T1, where nointermediate exposure enhancement is applied. Thus, 703 illustrates howthe pixel output signal of the four pixels will look at a normalexposure time.

Pixel plot 704 illustrates an output signal for the four pixels usingthe intermediate sample of the pixel, e.g., at 50% of the full exposuretime.

After applying the detail correction by detail correction circuit 200for the set of pixels P1 to P4, an output plot 704 illustrates theenhanced pixel output which more accurately reflects an emphasizedpresence of the moving object in pixels P2 and P3, (i.e., the object issensed by pixels P2 and P3 for 50% of the exposure duration, whilepixels P1 and P4 sense the object only for 25% of the exposureduration).

As shown in FIG. 7, for all pixels P1 to P4, the pixel charge betweensamples S3 and S2 (i.e., the intermediate exposure IE) is not equal toone half of the pixel charge between samples S4 and S1 (i.e., the fullexposure FE), indicating presence of motion at each of the pixels P1 toP4 in accordance with 606 where k=½. As a result, switch 507 iscontrolled to accept output from amplifier 205 in accordance with step610. Amplifier 205 enhances the pixel output for pixels P2 and P3 basedon the intermediate exposure between samples S3 and S2, and thus pixeloutput plot 704 reflects a higher amplitude than the pixel output plot703. For pixels P1 and P4, switch 507 takes output from amplifier 205(step 610), but as there is zero pixel charge accumulation at samples S3and S2, the pixel output calculated for intermediate exposure is zero inaccordance with step 604. As a result, the enhanced pixel output plot704 better reflects the image of the object at pixels P2 and P3 comparedto a stretched version across pixels P1, P2, P3 and P4.

FIG. 9 further illustrates this example. 902 illustrates a charge perpixel during throughout the length of a single exposure. The pixels aresampled at 4 samples during the exposure, S1, S2, S3, and S4. In FIG. 9,the first pixel sees the object only in the first 25% of the exposuretime, the second pixel in the first 50%, the third pixel in the second50% and the third pixel in the last 25%, but at the end of the exposure,the total exposure or output at S4 all show the same level.

First, it is considered at 904 whether the signal between the second,S2, and third, S3, samples is greater than or less than ½ of the fourthsample, S4, or total exposure output. This is true for each of thepixels. As this is true for each of the samples, motion is detected, anda difference between S2 and S3 may be used, e.g., with amplification, asthe output signal for the pixel.

Then, it is considered at 906 whether there is a difference between theoutput at S2 and S3. As there is no difference between S2 and S3 forpixels 1 and 4, the output for these pixels will be zero.

FIG. 8 shows an example motion blur reduction circuit 800 that combinesthe embodiments shown in FIG. 2A and FIG. 5A. In this example, detailcorrection signals generated with a reduced exposure time in the detailprocessor 206 may be added in summer 207 to the output 508 for fullexposure (FE) duration or the amplified output for the immediateexposure (IE) duration. The output 808 depends on the detection of amotion at step 606 by comparator 506 and the resulting output selectionat switch 507.

By way of example and without limitation, the aspects of the presentdisclosure are presented with reference to systems and methods used toconfigure various components of a video production system that may beused for production of television programming or at sports events. Thevarious concepts presented throughout this disclosure may be implementedacross a broad variety of imaging applications, including systems thatcapture and process video and/or still images, video conferencingsystems and so on.

The previous description is provided to enable any person skilled in theart to practice the various aspects described herein. Variousmodifications to these aspects will be readily apparent to those skilledin the art, and the generic principles defined herein may be applied toother aspects. Thus, the claims are not intended to be limited to theaspects shown herein, but is to be accorded the full scope consistentwith the language claims, wherein reference to an element in thesingular is not intended to mean “one and only one” unless specificallyso stated, but rather “one or more.” Unless specifically statedotherwise, the term “some” refers to one or more. All structural andfunctional equivalents to the elements of the various aspects describedthroughout this disclosure that are known or later come to be known tothose of ordinary skill in the art are expressly incorporated herein byreference and are intended to be encompassed by the claims. Moreover,nothing disclosed herein is intended to be dedicated to the publicregardless of whether such disclosure is explicitly recited in theclaims. No claim element is to be construed under the provisions of 35U.S.C. §112, sixth paragraph, unless the element is expressly recitedusing the phrase “means for” or, in the case of a method claim, theelement is recited using the phrase “step for.”

What is claimed is:
 1. A camera system for generating an image withimproved image sharpness by reducing motion blur of an object moving inthe image, the camera system comprising: an image sensor comprising anarray of pixels, with each pixel of the array being configured toaccumulate an electrical charge representative of the image capturedduring a frame; a pixel output sampler configured to sample a pixeloutput for each of the pixels of the image sensor to generate anintermediate pixel output sampled during an intermediate portion of theframe and an end pixel output sampled at an end portion of the frame; animage correction signal generator configured to generate a correctionsignal from the sampled intermediate pixel output value to enhance theend pixel output of the frame of the captured image; a cumulative pixeloutput determination module configured to determine a cumulative pixeloutput for each of the pixels of the array based on the sampled endpixel output of the frame; and an image signal output module configuredto apply the correction signal to the cumulative pixel output to producean enhanced pixel output for generating a digital video output for thecaptured image with reduced motion blur and judder.
 2. The camera systemof claim 1, wherein the pixel output sampler is further configured tosample a plurality of intermediate pixel outputs during respectiveintermediate portions, and the image correction signal generatorgenerates the correction signal based on the sampled plurality ofintermediate pixel outputs.
 3. The camera system of claim 1, wherein thepixel output sampler is further configured to sample the pixel output ata beginning portion of the frame, and for each of the pixels, thecumulative pixel output determination module determines the cumulativepixel output by subtracting the pixel output sampled at the beginningportion from the sampled end pixel output of the frame.
 4. The camerasystem of claim 3, wherein the image correction signal generator isfurther configured to: amplify a magnitude of the sampled intermediatepixel value by a magnitude that balances with a magnitude of thecumulative pixel output; perform detail processing on the amplifiedmagnitude to produce the correction signal; and add the correctionsignal to the cumulative pixel output to produce the enhanced pixeloutput.
 5. A camera, comprising: an image sensor comprising an array ofpixels, with each pixel being configured to accumulate an electricalcharge representative of an image captured during a frame; and a pixelprocessor configured to: sample a pixel output for at least one of thepixels of the image sensor during an intermediate portion of the frameto produce a signal representative of the image, and detect an object inmotion in the image by comparing a magnitude of the sampled pixel outputfor the intermediate portion of the frame to a fraction k of a magnitudeof a cumulative pixel output for the frame, wherein the fraction kequals a ratio of a duration between two sampled pixel outputs obtainedduring the intermediate portion of the frame and a full exposureduration of the frame.
 6. The camera of claim 5, wherein the pixelprocessor is further configured to produce the signal representative ofthe image by: selecting sampled pixel outputs obtained at the beginningand end of a first frame when no motion is detected; and selecting atleast one sampled pixel output obtained during the intermediate portionof the frame and amplifying the magnitude of the selected at least onesampled pixel output during the intermediate portion of the frame by amagnitude that balances with a magnitude of the sampled pixel output atthe end of a second frame when motion is detected.
 7. The camera ofclaim 6, wherein the pixel processor is further configured to generatedigital image data from the array of pixels such that a full image forone frame duration includes a mix of pixels having selected sampledpixel outputs when no motion is detected and selected sampled pixeloutputs when motion is detected.
 8. The camera of claim 6, wherein thepixel processor is further configured to: amplify a magnitude of thesampled pixel output during the intermediate portion of the frame by amagnitude that balances with the magnitude of the cumulative pixeloutput for the frame; perform detail processing on the amplifiedmagnitude to produce a detail correction signal; add the detailcorrection signal to the cumulative pixel output for the full frame toproduce digital image data corrected for motion blur when the object inmotion in the frame is not detected; and add the detail correctionsignal to the sampled pixel output of the intermediate portion of theframe when the object in motion in the frame is detected.
 9. The camerasystem of claim 1, wherein the image signal output module is furtherconfigured to determine digital image data for a plurality of adjacentpixels of the array of pixels for the object in motion that moves acrossthe plurality of adjacent pixels during one frame duration.
 10. Thecamera system of claim 1, wherein the pixel output sampler is furtherconfigured to sample each pixel of the image sensor during theintermediate portion of the frame occurring at one of: a first quarterand a third quarter of the frame; or a first third and a second third ofthe frame.
 11. The camera system of claim 1, wherein the imagecorrection signal generator is further configured to select at least onesample during the intermediate portion of the frame by selecting a pixeloutput having a least amount of blur according to a weighted detailcomparison of a plurality of pixel outputs sampled during the intermediaportion of the frame.
 12. A camera system for generating an image withimproved image sharpness by reducing motion blur of an object moving inthe image, the camera system comprising: an image sensor comprising anarray of pixels, with each pixel of the array being configured toaccumulate an electrical charge representative of the image capturedduring a frame; a pixel processor configured to: detect motion in theimage based on a charge accumulation rate of at least one pixel of thearray of pixels; sample a pixel output for the at least one pixel on acondition that motion is detected to generate an intermediate pixeloutput sampled during an intermediate portion of the frame and an endpixel output sampled at an end portion of the frame; generate acorrection signal from the sampled intermediate pixel output value toenhance the end pixel output of the frame of the captured image;determine a cumulative pixel output for the at least one pixel based onthe sampled end pixel output of the frame; and apply the correctionsignal to the cumulative pixel output to produce an enhanced pixeloutput for the captured image.
 13. The camera system of claim 12,wherein the pixel processor is further configured to sample a pixeloutput for each of the pixels of the image sensor during theintermediate portion of the frame.
 14. The camera system of claim 13,wherein the pixel processor is further configured to sample the pixeloutput of each of the pixels at a beginning portion of the frame, andfor each of the pixels, determine the cumulative pixel output bysubtracting the pixel output sampled at the beginning portion from thesampled end pixel output of the frame.
 15. The camera system of claim14, wherein the pixel processor is further configured to: amplify amagnitude of the sampled pixel output during the intermediate portion ofthe frame by a magnitude that balances with a magnitude of thecumulative pixel output for the frame; perform detail processing on theamplified magnitude to produce the correction signal; and add thecorrection signal to the cumulative pixel output to produce the enhancedpixel output.
 16. The camera system of claim 12, wherein the pixelprocessor is further configured to generate digital image data from thearray of pixels such that a full image for one frame duration includes amix of pixels having selected samples when no motion is detected andselected samples when motion is detected.
 17. The camera system of claim12, wherein the pixel processor is further configured to determine adigital image for a plurality of adjacent pixels of the array of pixelsfor an object in motion that moves across the plurality of adjacentpixels during one frame duration.
 18. A camera system for generating animage with improved image sharpness by reducing motion blur of an objectmoving in the image, the camera system comprising: an image sensorhaving a plurality of pixels, with each pixel being configured toconvert an image to a pixel output during a captured frame; a samplerconfigured to sample a pixel output for each of the pixels of the imagesensor during an intermediate portion of the frame and at an end portionof the frame; an intermediate sample subtractor configured to produce anintermediate exposure pixel output based on two samples of the pixeloutput sampled during the intermediate portion of the frame; a fullexposure subtractor configured to produce a cumulative pixel outputbased on an end pixel output sampled at the end portion of the frame; adetail correction signal generator configured to generate a detailcorrection signal from the intermediate exposure pixel output to enhancea digital video output for the captured image; and a detail correctionmodule configured to perform detail correction for the digital videooutput by applying the generated detail correction signal to thecumulative pixel output to produce an enhanced pixel output forgenerating the digital video output for the captured image with reducedmotion blur and judder.
 19. The camera system of claim 18, furthercomprising: a plurality of intermediate exposure subtractors configuredto determine a set of intermediate exposure pixel outputs; and anintermediate sample selector configured to select at least oneintermediate exposure pixel output from the set of intermediate exposurepixel outputs.
 20. A camera system for generating an image comprising:an image sensor comprising an array of pixels, with each pixel of thearray configured to accumulate an electrical charge representative ofthe image captured during a frame; a pixel output sampler configured tosample a pixel output for each of the pixels of the image sensor duringa beginning portion of the frame, a plurality of intermediate portionsof the frame, and at an end portion of the frame; a full frame exposureoutput configured to generate a full frame pixel output based on adifference between the sampled pixel output at the end portion of theframe and the sampled pixel output at the beginning portion of theframe; an intermediate frame exposure output configured to generate anintermediate exposure pixel output based on a difference between atleast a pair of sampled pixel outputs sampled during the plurality ofintermediate portions of the frame, respectively; an amplifierconfigured to amplify the generated intermediate exposure pixel outputas a normalization for a full frame exposure; a detail processorconfigured to generate a detail correction signal from the generated andamplified intermediate exposure pixel output; and an image signal outputmodule configured to apply the generated detail correction signal to thefull frame pixel output to produce an enhanced pixel output forgenerating a digital video output for the image with reduced motion blurand judder.
 21. The camera system according to claim 20, furthercomprising a comparator configured to determine a ratio of full exposureduration to intermediate exposure duration based on a first timeinterval between the beginning and end portions of the frame and asecond interval between a pair of the plurality of intermediate portionsof the frame.
 22. The camera system according to claim 21, wherein thecomparator is configured to determine whether the image contains motionbased on the determined ratio of the full exposure duration to theintermediate exposure duration.
 23. The camera system according to claim22, further comprising a switch controlled by the comparator andcoupling the image signal output module to one of the detail processorand the full frame exposure output based on a state of the switch. 24.The camera system according to claim 23, wherein the comparator sets theswitch to a first state coupling the image signal output module to thefull frame exposure output when no motion is detected in the image, suchthat image signal output module outputs the full frame pixel output forgenerating the digital video output for the image.
 25. The camera systemaccording to claim 24, wherein the comparator sets the switch to asecond state coupling the image signal output module to the detailprocessor when motion is detected in the image, such that the generateddetail correction signal is applied to the full frame pixel output toproduce the enhanced pixel output for generating the digital videooutput for the image with reduced motion blur and judder.